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(54) Electronic watermark insertion device 

(57) The present invention relates to an electronic 
watermark burying device that does not bury the same 
electronic watermark data in a superimposing mode. An 
electronic watermark insertion device comprises dis- 
crete cosine transforming means (120) for subjecting an 
input image into a discrete cosine transformation; elec- 
tronic watermark burying means (130) for inserting 
electronic watermark data to data transformed by the 
discrete cosine transforming means; electronic water- 
mark detecting means (180) for detecting whether or 
not the electronic watermark data to be inserted by the 
electronic watermark burying means (130) has been 
already inserted into the input image; and selecting 



means (180) for selecting the output data from the dis- 
crete cosine transforming means (120) when the elec- 
tronic watermark detecting means (1 80) detects that the 
electronic watermark data has been already inserted 
into the input image or for selecting output data from the 
electronic watermark burying means (130) when the 
electronic watermark detecting means (180) does not 
detect that the electronic watermark data has been not 
inserted into the input image; and means (150) for sub- 
jecting data selected by the selecting means into an 
inverse discrete cosine transformation. 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to an electronic 
watermark insertion device suitable for digital images, 
and particularly to an electronic watermark insertion 
device for inserting electronic watermark into digital 
images. 

[0002] Recently, illegal copies of digital images have 
become an important problem. Unlike analog images, 
since digital images are recognized by 0s and 1s, they 
can be repeatedly replicated without any degradation in 
image quality. This inherent feature has caused great 
damage to digital image copyright holders. 
[0003] In order to prevent such illegal replication, it is 
considered to prepare a reproduction system that 
encrypts digital image data and has a valid secret 
decryption key whereby encrypted digital image data 
can be reproduced. However, after the encryption is 
once decoded, this system cannot prevent subsequent 
replication. 

[0004] In order to prevent digital images from being 
illegally used or replicated, the method has been con- 
sidered of burying special information (hereinafter 
referred to as "electronic watermark data") in a digital 
image itself. 

[0005] Two types of data including visible electronic 
watermark data and invisible electronic watermark data 
are considered as electronic watermark data for digital 
images. 

[0006] The visible electronic watermark data, which 
contains special characters or symbols combined with 
an image, can be visually sensed. This electronic water- 
mark data may degrade the image quality but has the 
advantage of visually warning users to prevent misap- 
propriation of digital images. 

[0007] An example of burying such visible electronic 
watermark data is disclosed in JP-A-24 1403/1 996. This 
patent publication discloses a method of placing a visi- 
ble watermark on a digital image. This method consists 
of the steps of supplying an original digital image, sup- 
plying a digital watermark image, and superimposing 
the watermark on the original image, without changing 
the chromaticity of pixels of the original image upon 
watermarking, to create a watermarked image. This 
method is characterized in that the image-creating step 
includes the step of varying the brightness (not color) of 
respective opaque pixels in a watermarked image to 
correct pixels corresponding to the original image. In 
this method, only the brightness of pixels corresponding 
to opaque portions of electronic watermark data is var- 
ied so that visible electronic watermark data is synthe- 
sized with the original image without changing the color 
components. The scaling value of varying the pixel 
brightness component depends on color components, 
random numbers, pixel values of electronic watermark 
data, or others. The prominence of the watermark is 



determined by the scaling value. 
[0008] In invisible electronic watermark data, elec- 
tronic watermark data is buried in an image, in consider- 
ation of degradation of image quality. Since the image 
5 quality degradation is not substantially negligible, the 
watermark cannot be visually recognized. 
[0009] As described above, since special information, 
which can be recognized by an author, is buried as the 
electronic watermark data, the author can be specified 
w by detecting the electronic watermark data even after 
illegal replication. Moreover, information about replica- 
tion disapproval may be buried in an image. In such a 
case, when the reproduction unit, for example, detects 
the replication disapproval information, the reproduction 
75 by a VTR or the equivalent can be restricted by inform- 
ing the user that the detected information is reproduc- 
tion prohibited data, or by operating the replication 
preventing mechanism within the reproduction unit. 
[0010] As one method of burying invisible electronic 
so watermark data into digital images, special information 
is buried as electronic watermark data in portions not 
substantially affecting the image quality, such as LSBs 
(least significant bits) of pixel data. However, according 
to this method, the electronic watermark data can be 
25 easily removed from images. For example, information 
regarding LSBs of pixels will be missed using a low- 
pass filter. The image compression process discards 
the volume of information not adversely affecting the 
image quality, thus reducing the volume of data. This 
30 means that the electronic watermark data is lost. As a 
result, the problem is that it is difficult to re-detect the 
electronic watermark data. 

[0011] JP-A-No. 315131/1994 shows another exam- 
ple of the electronic watermark burying method. This 
35 publication discloses the method of burying specific 
information by using the correlation between continuous 
frame images and detecting the area where degrada- 
tion in image quality does not occur even when substitu- 
tion is performed in peripheral areas upon reproduction. 
40 According to this method, an image is reconstituted by 
specifying an identification data buried area using the 
signal dropout portion and conversion information upon 
reproduction and then by correcting the corresponding 
portion. 

45 [0012] As further another example, JP-A-No. 
30466/1993 discloses the method of converting the fre- 
quency of a video signal and then burying information 
with signals of frequencies lower than the frequency 
band of the converted video signal. In this method, a 
so broad band-pass filter extracts the original video signal 
while a low-pass filter extracts the buried identification 
data. 

[0013] In another example, the method of frequency- 
converting images and then burying electronic water- 
55 mark data into portions with strong frequency compo- 
nents of a video signal after the frequency conversion 
(see "NIKKEI Electronics". 1996, 4.22 (no. 660), page 
13). In this method, since electronic watermark data is 
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buried into frequency components, the electronic water- 
mark data is not lost through the compression process 
or filtering image process. Moreover, using the random 
numbers with a normal distribution as electronic water- 
mark data makes it difficult to prevent interference 
between electronic watermark data and to destroy the 
electronic watermark data without significantly affecting 
the entire image. 

[0014] In the electronic watermark data burying 
method, the original image is first transformed into fre- 
quency components by the DCT (discrete cosine 
transformation) 703. n pieces of data with high values 
over high frequency range are selected as f(l), f(2), , 
f(n). The electronic watermark data, w(1), w(2), • w(n), 
are selected from a normal distribution having an aver- 
age of 0 and a dispersion of 1. The formula, 
F(i) = f(i) + a x |f(i)| x w(i) , where a is a scaling factor, 
is calculated to obtain respective (i)s. Finally, the image 
in which the electronic watermark data is buried is 
obtained based on the frequency component in which 
f(i) is substituted for F(i). 

[0015] Moreover, the electronic watermark data is 
detected according to the following method. In this 
detection method, both the original image and elec- 
tronic watermark data candidate w(i) (where i = 1, 2, . 
n) must be known. 

[0016] First, the image containing electronic water- 
mark data is converted into frequency components 
through, for example, DCT. Values corresponding to fac- 
tor values, f(1), f(2), -, f(n), each containing an elec- 
tronic watermark, are set as F(1), F(2), , F(n), 
respectively. The formula, W(i) = (F(i) - f(i)) / f(i) , is 
solved using f(i) and F(i) to extract the electronic water- 
mark data W(i). Next the statistical similarity C between 
w(i) and W(i) is obtained by the following formula includ- 
ing a vector inner product. 

C = W • w/ (WD x wD) 

where W = (W(1), W(2), W(n); w = (w(1), w(2), , 
w(n)); WD = the absolute value of a vector W; wD = the 
absolute value of a vector w; and the symbol • repre- 
sents an inner product. 

[0017] When the statistical similarity C is more than a 
specific value, it is judged that the electronic watermark 
data is in a buried state. 

[0018] The above-mentioned method, where the elec- 
tronic watermark data is buried into an image, is effec- 
tive when an author holding an original image detects 
digital image data suspected as an illegal replicate. 
[0019] In the above-mentioned method that requires 
an original image, the author, that is, an original image 
owner, can detect image data doubted as an illegal rep- 
licate. However, the reproduction unit in each terminal 
cannot detect electronic watermark data because of the 
absence of the original image. 

[0020] To overcome that problem, an improvement of 
the above-mentioned method for the terminal process- 



ing has been proposed. In the improved method, the 
original image is divided into blocks each having K pix- 
els x K pixels. Electronic watermark data is buried or 
extracted in block process units. 

5 [0021 ] In the electronic watermark data burying proc- 
ess, AC frequency components are set as f(1), f(2), 
f(n) in a frequency increasing order over a frequency 
range after the discrete cosine transformation. The 
electronic watermark data, w(1), w(2), w(n) are 

10 selected from the normal distribution having an average 
of 0 and a dispersion of 1 . In order to obtain respective 
(i)s, the formula of F(i) = f(i) + a x avg (f(i)) x w(i) is 
calculated, where a is a scaling factor and avg(f(i)) is a 
partial average obtained by averaging the absolute val- 

75 ues at three points adjacent to f(i). An image in which 
electronic watermark data is buried can be obtained 
from the frequency components where f (i) is substituted 
with F(i). 

[0022] Electronic watermark data is detected accord- 
20 ing to the following method. This method does not 
require any original image. It is merely required that 
electronic watermark data candidate w(i) (where i = 1 , 2, 
, n) is known. 

[0023] Over the frequency band where electronic 
25 watermark data contained image is subjected to a dis- 
crete cosine transformation, the frequency components 
are set as F(1), F(2), , F(n) in a frequency increasing 
order. The average of the absolute values of three adja- 
cent points in the watermark data F(i) is set to a partial 
30 average avg(F(i)). The electronic watermark data W(i) is 
obtained by calculating the following formula 

W(i) = F(i)/avg(F(i)) 

35 [0024] Moreover, the sum WF(i) is obtained by calcu- 
lating w(i) for each frame according to the above-men- 
tioned formula. Next the statistical similarity C between 
w(i) and WF(i) is obtained by calculating the following 
formula including an vector inner product: 

40 

C = WF • w/(WFD x wD) 

where WF = (WF(1), WF(2), , WF(n)); w = (w(1), w(2), 
, w(n)); WFD = the absolute value of a vector WF; wD 
45 = the absolute value of a vector w; and the symbol • 
represents an inner product. When the statistical simi- 
larity C is more than a specif ic value, it is judged that the 
corresponding electronic watermark data is in a buried 
state. 

so [0025] Fig. 3 shows a prior art electronic watermark 
inserting device employing the above-mentioned 
method. Referring to Fig. 3, the input image 310 is 
transformed from the time region to the frequency 
region by means of the discrete cosine transformation 

55 unit 320. The electronic watermark burying unit 320 
inserts the electronic watermark data 340 in the result- 
ant data. The buried electronic watermark data is trans- 
formed from the frequency region to the time region by 
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means of the inverse discrete cosine transformation unit 
350. Thus, the output image 360 with the inserted elec- 
tronic watermark data can be obtained. 
[0026] However, the prior -art device may insert evenly 
and repeatedly electronic watermark data even in an s 
image with the same. 

[0027] Fig. 4 schematically shows the concept of bur- 
ying electronic watermark data in a frequency region. 
The frequency spectrum 440 after the electronic water- 
mark data burying operation is formed by adding the f re- 10 
quency spectrum 420 of electronic watermark data to 
the frequency region of an input image. Allocating a 
specific region of an input image does not depend on 
the type of electronic watermark data. When the elec- 
tronic watermark data burying process is performed plu- is 
ral times, the same electronic watermark data is again 
buried on the frequency spectrum of an image with data 
previously buried. If the same electronic watermark data 
is buried twice or three times, the upper black portion 
corresponding to the buried amount becomes larger in 20 
the frequency spectrum 440' after the burying process. 
When the frequency region of the spectrum is con- 
verted into the time region, the burying noises become 
larger, so that images are emphasized with the noise 
components. 25 
[0028] Each piece of electronic watermark data is 
inserted as very feeble noises, compared w;+h the origi- 
nal image and is not visually recognized by a human 
eye. However, there is the problem in that if noises 
repeatedly integrated are large, it may be sensed by the 30 
human eye, thus leading to degradation in image qual- 
ity. 

SUMMARY OF THE INVENTION 

35 

[0029] The present invention is made to solve the 
above-mentioned problems. 

[0030] Moreover, the objective of the invention is to 
provide an electronic watermark data burying device 
that can bury the same electronic watermark data into 40 
an image only once, not duplicative! y. 
[0031] The objective of the present invention is 
achieved by an electronic watermark insertion device 
comprising discrete cosine transforming means for sub- 
jecting an input image to a discrete cosine transforma- 45 
tion; electronic watermark burying means for inserting 
electronic watermark data into data transformed by the 
discrete cosine transforming means; electronic water- 
mark detecting means for detecting whether or not the 
electronic watermark data to be inserted by the elec- so 
tronic watermark burying means has been already 
inserted into the input image; and selecting means for 
selecting the output data from the discrete cosine trans- 
forming means when the electronic watermark detect- 
ing means detects that the electronic watermark data ss 
has been already inserted into the input image or for 
selecting output data from the electronic watermark bur- 
ying means when the electronic watermark detecting 



means does not detect that the electronic watermark 
data has been inserted into the input image; and means 
for subjecting data selected by the selecting means to 
an inverse discrete cosine transformation. 
[0032] In the electronic watermark insertion device 
according to the present invention, the electronic water- 
mark detecting means decides that the corresponding 
electronic watermark data is buried, when a statistical 
similarity C is more than a specific value. 
[0033] Moreover, in the electronic watermark insertion 
device according to the present invention, the elec- 
tronic watermark detecting means first orthogonally 
converts an image with the electronic watermark data 
into a frequency component; calculates and extracts 
electronic watermark data W(i) using the formula 
W(i) s (F(i) - f(i)) / f(i) , where both an original image 
and electronic watermark data candidate wi (where i = 
1, 2, ■ n) are known; where f(1), f(2), -f(n) to which 
electronic watermarks are respectively buried in a 
frequency range correspond factor values F(1), 
F(2), F(n); next calculates a statistical similarity C 
between w(i) and W(i) using the formula, 
C - W • w / (WD x wD) , where W = (W(1), W(2), 
W(n)), w = (w(1), w(2), w(n)), WD = the absolute value 
of a vector W, and wD = the absolute value of a vector 
w), and the symbol • represents a vector inner product; 
and detects that the electronic watermark data is in a 
buried state when the statistical similarity C is more than 
a specific value. 

[0034] In the electronic watermark insertion device 
according to the present invention, the electronic water- 
mark burying means sets AC frequency components 
(f(1), f(2), f(n)) in a frequency increasing order over a 
frequency range after a discrete cosine transformation; 
selects electronic watermark data, w(1), w(2) t , w(n), 
from normal distribution having an average of 0 and a 
dispersion of 1 ; and calculates respective (i)s the for- 
mula, F(i) = f(i) + a x |f(i)| x w(i) (where a is a scaling 
factor) so that an image, in which the electronic water- 
mark data is buried, is obtained from the frequency 
component in which f(i) is substituted with F(i). 
[0035] Moreover, in the electronic watermark insertion 
device according to the present invention, the electronic 
watermark burying means sets AC frequency compo- 
nents (f(1), f(2), . f(n)) in a frequency increasing order 
over a frequency range after a discrete cosine transfor- 
mation; selects electronic watermark data, w(1), w(2), 
, w(n). from normal distribution having an average of 0 
and a dispersion of 1; and calculates respective (i)s 
using the formula, F(i) = f(i) + a x avg |f(i)| x w(i) 
(where a is a scaling factor and avg f(i) is a partial aver- 
age of absolute values of f(i) at three adjacent points) so 
that an image, in which the electronic watermark data is 
buried, is obtained from the frequency component in 
which f(i) is substituted with F(i). 
[0036] Moreover, according to the present invention, 
an electronic watermark insertion device comprises 
orthogonal converting means for subjecting an input 
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image to an orthogonal conversion; electronic water- 
mark burying means for inserting electronic watermark 
data to data converted by the orthogonal converting 
means; electronic watermark detecting means for 
detecting whether or not the electronic watermark data 5 
to be inserted by the electronic watermark burying 
means has been already inserted into the input image; 
selecting means for selecting the output data from the 
orthogonal converting means when the electronic 
watermark detecting means detects the electronic w 
watermark data already inserted into the input image or 
for selecting output data from the electronic watermark 
burying means when the electronic watermark detect- 
ing means does not detect the electronic watermark 
data inserted into the input image; and means for sub- 75 
jecting data selected by the selecting means into an 
inverse discrete cosine transformation. 
[0037] Furthermore, according to the present inven- 
tion, an electronic watermark insertion device com- 
prises orthogonal converting means for subjecting an 20 
input image to an orthogonal conversion; electronic 
watermark detecting means for detecting whether or not 
electronic watermark data to be inserted has been 
already inserted into the input image; electronic water- 
mark data selecting means for selecting electronic 25 
watermark data by which the input image is not affected 
in data insertion when the electronic watermark detect- 
ing means detects the electronic watermark data 
already inserted into the input image or for selecting the 
electronic watermark data to be inserted when the elec- 30 
tronic watermark detecting means does not detect the 
electronic watermark data inserted into the input image; 
electronic watermark burying means for inserting elec- 
tric watermark data selected by the electronic water- 
mark data selecting means into data converted by the 35 
orthogonal converting means; and means for subjecting 
data into which the electronic watermark data is buried 
by the electronic watermark burying means, to an 
inverse orthogonal conversion. 

[0038] In the electronic watermark insertion device 40 
according to the present invention, the electronic water- 
mark detecting means decides that the corresponding 
electronic watermark data is buried, when a statistical 
similarity C is more than a specific value. 
[0039] In the electronic watermark inserting device 45 
according to the present invention, the orthogonal con- 
verting means performs discrete Fourier transformation, 
discrete cosine transformation, Hadamard- Walsh trans- 
formation, or Karhunen-Loeve transformation. 

50 

BRIEF DESCRIPTION OF THE INVENTION 

[0040] This and other objects, features, and advan- 
tages of the present invention will become more appar- 
ent upon a reading of the following detailed description ss 
and drawings, in which: 

Fig. 1 is a block diagram illustrating an electronic 



watermark data insertion device according to a first 
embodiment of the present invention; 
Fig. 2 is a block diagram illustrating an electronic 
watermark data insertion device according to a sec- 
ond embodiment of the present invention; 
Fig. 3 is a block diagram illustrating a prior-art elec- 
tronic watermark data insertion device; and 
Fig. 4 is a diagram explaining a method of burying 
electronic watermark data over a frequency band. 

DESCRIPTION OF THE EMBODIMENTS 

[0041 ] Embodiments of the present invention will be 
described below with reference to the attached draw- 
ings. 

(First Embodiment) 

[0042] The first embodiment of the present invention 
will be now described with reference with the drawings. 
Referring to Fig. 1 , the electronic watermark data inser- 
tion device consists of an input image 110, a discrete 
cosine transformation (DCT) unit 120 for transforming 
an input image from a time region to a frequency region, 
an electronic watermark inserting unit 130 for inserting 
electronic watermark data into data converted into a fre- 
quency region by means of the discrete cosine transfor- 
mation unit 120, electronic watermark data 140 to be 
inserted by the electronic watermark burying unit 130, 
an electronic watermark detector 180 for * detecting 
whether or not the electronic watermark data 140 is 
inserted into data output from the discrete cosine trans- 
formation unit 120, a buffer 170 for temporarily storing 
data output from the discrete cosine transformation unit 
1 20, a selector 1 90 for selecting data from the electronic 
watermark burying unit 130 or data from the buffer 170 
in response to the output from the electronic watermark 
detector 180, an inverse discrete cosine transformation 
(inverse DCT) unit 150 for transforming data from the 
selector 190 from a frequency region to a time region, 
and an output image 160 to be output from the inverse 
discrete cosine transformation unit 150. 
[0043] Next, the operation of the electronic watermark 
data burying device shown in Fig. 1 will be described 
with reference to the drawings. 

[0044] The discrete cosine transformation (DCT) unit 
120 orthogonally converts the input image 110 from a 
time region to a frequency region. The converted data is 
input to the electronic watermark burying unit 130, the 
electronic watermark detector 180 and the buffer 170. 
[0045] The electronic watermark burying unit 130 
inserts electronic watermark data 140 to the converted 
data. At the same time, the electronic watermark detec- 
tor 180 detects whether or not the electronic watermark 
data 140 is in an inserted state in the input data. The 
buffer 170 temporarily stores the output data of the dis- 
crete cosine transformation unit 120. 
[0046] The selector 190 selects either the output data 
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from the electronic watermark burying unit 130 or the 
output data from the buffer 170 based on the detection 
result of the electronic watermark detector 180. That is, 
the selector 190 selects the output data from the elec- 
tronic watermark burying unit 130 when the electronic 
watermark detector 180 does not detect the output data 
and selects the output data from the buffer 170 when 
the electronic watermark detector 180 detects the out- 
put data. 

[0047] The data selected by the selector 190 corre- 
sponds to the output image 160 transformed from the 
frequency region to the time region by the inverse 
cosine transformation unit 150. 

[0048] As described above, only when the input image 
110 does not contain the electronic watermark data 
140, the electronic watermark data 140 is inserted in 
the input image 110. 

[0049] In this embodiment, the buffer 170 is used to 
synchronize two data to be selected by the selector 1 90 
with each other, in consideration of the process time of 
the electronic watermark burying unit 130. If the proc- 
ess time of the electronic watermark burying unit 130 is 
practically and negligibly short, the buffer 170 can be 
omitted. 

(Second Embodiment) 

[0050] Next, the second embodiment of the present 
invention will be described with reference to the 
attached drawings. 

[0051] Referring to Fig. 2, the electronic watermark 
data insertion device consists of an input image 210, a 
discrete cosine transformation (DCT) unit 220 for trans- 
forming an input image from a time region to a fre- 
quency region, electronic watermark data B245 to be 
inserted, electronic watermark detector 280 for detect- 
ing whether or not the electronic watermark data 245 is 
inserted into data output from the discrete cosine trans- 
formation unit 220, an electronic watermark data selec- 
tor 290 for selecting the electronic watermark data A240 
or the electronic watermark data B245 based on the 
detection result from the electronic watermark detector 
280, an electronic watermark burying unit 230 for insert- 
ing electronic watermark data selected by the electronic 
watermark data selector 290 into data transformed into 
a frequency region by means of the discrete cosine 
transformation unit 220, an inverse discrete cosine 
transformation (inverse DCT) unit 250 for transforming 
data output from the electronic watermark burying unit 
230 from the frequency region to the time region, and an 
output image 260 to be output from the inverse discrete 
cosine transformation unit 250. 

[0052] Next, the operation of the electronic watermark 
data burying unit will be described with reference to Fig. 
2. 

[0053] The discrete cosine transformation (DCT) unit 
220 orthogonally transforms the input image 210 from 
the time region to the frequency region. The trans- 



formed data is input to the electronic watermark burying 
unit 230 and the electronic watermark detector 280. 
[0054] The electronic watermark detector 280 detects 
whether or not there is the electronic watermark data 

5 B245 inserted into the input data. The electronic water- 
mark data selector 290 selects electronic watermark 
data based on the detection result of the electronic 
watermark detector 280. That is. when the electronic 
watermark data detector 280 does not detect electronic 

10 watermark data B245, the electronic watermark data 
selector 290 selects it. When the electronic watermark 
data detector 280 detects electronic watermark data, 
the electronic watermark data selector 290 selects the 
electronic watermark data A245. 

is [0055] Because the input image converted into a fre- 
quency region is detected, it is not needed to bury 
watermark information all over the space range of the 
input image. However, where a watermark image is bur- 
ied in a part of divided blocks of the input image, the 

20 missing information about a block can be restored with 
information about another block. 

[0056] The electronic watermark burying unit 230 
inserts electronic watermark data selected by the elec- 
tronic watermark data selector 290 into an image. The 

25 inverse discrete cosine transformation unit 250 trans- 
form s the output data from the electronic watermark 
burying unit 230 from the frequency region to the time 
region and then outputs the output image 260. 
[0057] Inputting the electronic watermark data A240 

30 with no frequency components does not adversely 
affect the input image. Hence, the input image which 
previously contains the electronic watermark data B245 
is not substantially subjected to the electronic water- 
mark data insertion and is output without any change. 

35 [0058] In this embodiment, it is assumed that the proc- 
ess time of the electronic watermark detector 280 and 
the data selector 290 is negligibly short. If the process 
time is not negligible, a buffer is connected to the input 
of the electronic watermark burying unit 230. The buffer 

40 temporarily stores the output data of the discrete cosine 
transformation unit 220 and then inputs it to the elec- 
tronic watermark data detector 230. Thus the process 
time of the electronic watermark detector 280 and the 
electronic watermark data selector 290 can be can- 

45 celed. 

[0059] Where the block coordinates are known, the 
electronic watermark detection can be performed 
according to the above-mentioned embodiment It is not 
necessary that the spatial block coordinates are known. 

so However, it is needed that the block size (k pixels x k 
pixels), conversion rule to frequency regions, electronic 
watermark data, and the like are known. Such informa- 
tion are predetermined between the electronic water- 
mark inserting device and the detector. If the 

55 information is leaked to the third party, data may be ille- 
gally replicated. 

[0060] In the above-mentioned embodiment, original 
images are subjected to a discrete cosine transforma- 
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tion (DCT). The frequency spectrum may be resolved by 
performing orthogonal conversion including Hadamard 
transformation (Hadamard- Walsh transformation), 
wavelet transformation, multiple resolution, sub-band 
method, discrete Fourier transformation, Haar transfor- 
mation, Karhunen-Loeve transformation, gradient Had- 
amard transformation, and the like. 
[0061 ] Each above-mentioned embodiment shows the 
example of burying invisible electronic watermark data. 
However, the concept of the invention is not impaired by 
burying visible electronic watermark data. 
[0062] Moreover, in each embodiment, the scaling 
parameter can be selected from scaling parameters that 
add an additional watermark value that adversely 
affects the visual quality of data. The watermark value 
can be selected according to the normal distribution. 
[0063] As described above, the present invention has 
the advantage in that since the same electronic water- 
mark data is not buried in an image plural number of 
times, degradation in image quality can be avoided. 
[0064] Moreover, the present invention can deal with 
electronic watermark data visually or invisibly. Hence, 
the degradation in image quality of the particular portion 
where the same electronic watermark data is overwrit- 
ten can be avoided by detecting the presence or 
absence of the same electronic watermark data within 
an image with reference electronic watermark data. 
Even the original image copyright holder cannot insert 
electronic watermark data to the original image twice or 
more so that the author personality right can be sub- 
stantially maintained validly. 

Claims 

1. An electronic watermark insertion device, compris- 
ing: 

discrete cosine transforming means (120) for 
subjecting an input image to a discrete cosine 
transformation; 

electronic watermark burying means (130) for 
inserting electronic watermark data into data 
transformed by said discrete cosine transform- 
ing means (120); 

electronic watermark detecting means (1 80) for 
detecting whether or not said electronic water- 
mark data to be inserted by said electronic 
watermark burying means (180) has been 
already inserted into said input in* age; and 
selecting means (1 90) for selecting said output 
data from said discrete cosine transforming 
means (120) when said electronic watermark 
detecting means (130) detects that said elec- 
tronic watermark data has been already 
inserted into said input image or for selecting 
output data from said electronic watermark bur- 
ying means (130) when said electronic water- 
mark detecting means (180) does not detect 



that said electronic watermark data has been 
inserted into said input image; and 
means (150) for subjecting data selected by 
said selecting means (190) to an inverse dis- 
5 crete cosine transformation. 

2. The electronic watermark insertion device defined 
in Claim 1, wherein said electronic watermark 
detecting means (180) decides that the corre- 

w sponding electronic watermark data is buried, when 
a statistical similarity C is more than a specific 
value. 

3. The electronic watermark insertion device defined 
75 in Claim 1, said electronic watermark detecting 

means (180) first orthogonally converts an image 
with said electronic watermark data into a fre- 
quency component; calculates and extracts elec- 
tronic watermark data W(i) using the formula 

20 W(i) = (F(i) - f(i)) / f(i) , where both an original 
image and electronic watermark data candidate wi 
(where i = 1, 2/ n) are known; where f(1), f(2), 
f(n) to which electronic watermarks are respec- 
tively buried in a frequency range correspond factor 

25 values F(1), F(2), F(n); next calculates a statistical 
similarity C between w(i) and W(i) using the for- 
mula, C = W • w / (WD x wD) where W = (W(1), 
W(2), - W(n)), w = (w(1), w(2), - w(n)) ( WD = the 
absolute value of a vector W, and wD = the absolute 

30 value of a vector w, and the symbol • represents a 
vector inner product; and detects that said elec- 
tronic watermark data is in a buried state when said 
statistical similarity C is more than a specific value. 

35 4. The electronic watermark insertion device defined 
in Claim 1 , wherein said electronic watermark bury- 
ing means (130) sets AC frequency components 
(f(1), f(2), , f(n)) in a frequency increasing order 
over a frequency range after a discrete cosine 

40 transformation; selects electronic watermark 
data, w(1), w(2), , w(n), from normal distribution 
having an average of 0 and a dispersion of 1 ; 
and calculates respective (i)s the formula, 
F(i) = f(i) + a x |f(i)| x w(i) (where a is a scaling 

45 factor) so that an image, in which said electronic 
watermark data is buried, is obtained from the fre- 
quency component in which f(i) is substituted with 
F(i). 

so 5. The electronic watermark insertion device defined 
in Claim 1 , wherein said electronic watermark bury- 
ing means (130) sets AC frequency components 
(f(1), f(2), f(n)) in a frequency increasing order 
over a frequency range after a discrete cosine 

55 transformation; selects electronic watermark data, 
w(1), w(2), , w(n), from normal distribution having 
an average of 0 and a dispersion of 1 ; and calcu- 
lates respective (i)s using the formula, 
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7. 



F(i) = f(i) + a x avg | f(i)( x w(i) (where a is a scal- 
ing factor and avg f(i) is a partial average of abso- 
lute values of f(i) at three adjacent points) so that an 
image, in which said electronic watermark data is 
buried, is obtained from the frequency component 
in which f(i) is substituted with F(i). 

An electronic watermark insertion device compris- 
ing: 

discrete cosine transforming means (220) for 
subjecting an input image to a discrete cosine 
transformation; 

electronic watermark detecting means (280) for 
detecting whether or not electronic watermark 
data to be inserted has been already inserted 
into said input image; 

electronic watermark data selecting means 
(290) for selecting electronic watermark data 
by which said input image is not affected in 
data insertion when said electronic watermark 
detecting means (280) detects said electronic 
watermark data already inserted into said input 
image or for selecting said electronic water- 
mark data to be inserted when said electronic 
watermark detecting means (280) does not 
detect said electronic watermark data inserted 
into said input image; 

electronic watermark burying means (230) for 
inserting electric watermark data selected by 
said electronic watermark data selecting 
means (290) into data transformed by said dis- 
crete cosine transforming means (220) ; and 
inverse discrete cosine transforming means 
(250) for subjecting data into which said elec- 
tronic watermark data is buried by said elec- 
tronic watermark burying means (230), to an 
inverse discrete cosine transformation. 

The electronic watermark insertion device defined 
in Claim 6, wherein said electronic watermark 
detecting means (280) decides that the corre- 
sponding electronic watermark data is buried, when 
a statistical similarity C is more than a specific 
value. 

The electronic watermark insertion device defined 
in Claim 6, said electronic watermark detecting 
means (280) first orthogonally converts an image 
with said electronic watermark data into a fre- 
quency component; calculates and extracts elec- 
tronic watermark data W(i) using the formula 
W(i) = (F(i) - f(i)) / f(i) . where both an original 
image and electronic watermark data candidate wi 
(where i = 1.2, ... n) are known; where f (1), f (2), ... 
f(n) to which electronic watermarks are respectively 
buried in a frequency range correspond factor val- 
ues F(1), F(2) , ••• F(n); next calculates a statistical 



similarity C between w(i) and W(i) using the for- 
mula, C = W • w / (WD x wD) . where W = (W(1), 
W(2), ■ W(n)), w = (w(1), w(2), - w(n)), WD = the 
absolute value of a vector W, and wD = the absolute 
5 value of a vector w, and the symbol • represents a 

vector inner product; and detects that said elec- 
tronic watermark data is in a buried state when said 
statistical similarity C is more than a specific value. 

10 9. The electronic watermark insertion device defined 
in Claim 6, wherein said electronic watermark bury- 
ing means (230) sets AC frequency components 
(f(1), f(2), - , f(n)) in a frequency increasing order 
over a frequency range after a discrete cosine 

is transformation; selects electronic watermark 

data, w(1), w(2) w(n), from normal distribution 

having an average of 0 and a dispersion of 1 ; 
and calculates respective (i)s the formula, 
F(i) = f(i) + a x |f(i)| x w(i) (where a is a scaling 

20 factor) so that an image, in which said electronic 
watermark data is buried, is obtained from the fre- 
quency component in which f(i) is substituted with 
F(i). 

25 10. The electronic watermark insertion device defined 
in Claim 6, wherein said electronic watermark bury- 
ing means (230) sets AC frequency components 
(f(1), f(2), . f(n)) in a frequency increasing order 
over a frequency range after a discrete cosine 

30 transformation; selects electronic watermark 
data, w(1), w(2), -, w(n), from normal distribution 
having an average of 0 and a dispersion of 1 ; and 
calculates respective (i)s using the formula, 
F(i) = f(i) + a x avg |f(i)| x w(i) (where a is a scal- 

35 ing factor and avg f(i) is a partial average of abso- 
lute values of f(i) at three adjacent points) so that an 
image, in which said electronic watermark data is 
buried, is obtained from the frequency component 
in which f(i) is substituted with F(i). 



40 



45 



so 



55 



11. An electronic watermark insertion device, compris- 
ing: 

orthogonal converting means (1 20) for subject- 
ing an input image to an orthogonal conver- 
sion; 

electronic watermark burying means (130) for 
inserting electronic watermark data to data 
transformed by said orthogonal converting 
means (120); 

electronic watermark detecting means (180) for 
detecting whether or not said electronic water- 
mark data to be inserted by said electronic 
watermark burying means (190) has been 
already inserted into said input image: 
selecting means (1 90) for selecting said output 
data from said orthogonal converting means 
(120) when said electronic watermark detect- 
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ing means (1 80) detects said electronic water- 
mark data already inserted into said input 
image or for selecting output data from said 
electronic watermark burying means (130) 
when said electronic watermark detecting 5 
means (180) does not detect said electronic 
watermark data inserted into said input image; 
and 

means (150) for subjecting data selected by 
said selecting means into an inverse discrete w 
cosine transformation. 



12. The electronic watermark insertion device defined 
in Claim 11, wherein said electronic watermark 
detecting means (180) decides that the corre- is 
sponding electronic watermark data is buried, when 

a statistical similarity C is more than a specific 
value. 

13. The electronic watermark inserting device defined 20 
in Claim 11, wherein said orthogonal converting 
means (120) performs discrete Fourier transforma- 
tion, discrete cosine transformation, Hadamard- 
Walsh transformation, or Karhunen-Loeve transfor- 
mation 25 

14. An electronic watermark insertion device compris- 
ing: 

orthogonal converting means (220) for subject- 30 
ing an input image to an orthogonal conver- 
sion; 

electronic watermark detecting means (280) for 
detecting whether or not electronic watermark 
data to be inserted has been already inserted 35 
into said input image; 

electronic watermark data selecting means 
(290) for selecting electronic watermark data 
by which said input image is not affected in 
data insertion when said electronic watermark 40 
detecting means (280) detects said electronic 
watermark data already inserted into said input 
image or for selecting said electronic water- 
mark data to be inserted when said electronic 
watermark detecting means (280) does not 45 
detect said electronic watermark data inserted 
into said input image; 

electronic watermark burying means (230) for 
inserting electric watermark data selected by 
said electronic watermark data selecting so 
means (280) into data converted by said 
orthogonal converting means (220); and 
means (250) for subjecting data into which said 
electronic watermark data is buried by said 
electronic watermark burying means (230), to 55 
an inverse orthogonal conversion. 

15. The electronic watermark insertion device defined 



in Claim 14, wherein said electronic watermark 
detecting means (280) decides that the corre- 
sponding electronic watermark data is buried, when 
a statistical similarity C is more than a specific 
value. 

16. The electronic watermark inserting device defined 
in Claim 14, wherein said orthogonal converting 
means (220) performs discrete Fourier transforma- 
tion, discrete cosine transformation, Hadamard- 
Walsh transformation, or Karhunen-Loeve transfor- 
mation. 



35 



40 



45 



so 



BNSOOCID: <EP 0955767A2_I_> 



9 



EP 0 955 767 A2 



o 



O £ 



8 




w o 55 
> £2 O 
5 q o 



8 



o 

b 

UJ 
UJ 

cn 



CD 

LU 



LU 

u. 

LL 
3 
CD 



O 



o 

CO 



1 



2 1 ^ 

O LU >- 

uj h cr 

uj S m 



o * 

II 

UJ f h 
j § < 
UJSQ 




o 




BNSDOCID: <EP 0955767A2_I_> 



10 



EP 0 955 767 A2 



CM 

CD 




11 



EP 0 955 767 A2 





CO 

CD 

li- 



es 

CO 

n 



O * z 
2 g 3 

2 1 o 
o £ > 

ill rr 01 
£ Z> 
UJ > CD 



O 

5 



Pi 

LU > Q 





12 

DWCIWia --CD ru-»c: me?"? a -> i „ 



EP 0 955 767 A2 



FIG.4 



410 



430 



1 



FREQUENCY SPECTRUM OF 
INPUT IMAGE 



- 420 



LflLH. 





r 


nil 


iii, 



440 



FREQUENCY SPECTRUM 
AFTER BURYING ELECTRONIC 
WATERMARK DATA 



FREQUENCY SPECTRUM OF 
ELECTRONIC WATERMARK DATA 



BNSDOCID: <EP 095S767A2 I > 



13 



THIS PAGE BUWK^o) 



(19) 



J) 



(12) 



(88) Date of publication A3: 

30.05.2001 Bulletin 2001/22 

(43) Date of publication A2: 

10.11.1999 Bulletin 1999/45 

(21) Application number: 99108714.9 

(22) Date of filing: 30.04.1999 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets (11) EP 0 955 767 A3 

EUROPEAN PATENT APPLICATION 

(51) lnt.CI 7 : H04N 1/32 



(84) Designated Contracting States: 


(72) 


Inventor: Nakano, Hirotaka 


AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 




Minato-ku, Tokyo 108-01 (JP) 


MC NL PT SE 




Designated Extension States: 


(74) 


Representative: 


AL LT LV MK RO SI 




von Samson-Himmelstjerna, Friedrich R M Dipl.- 
Phys. et al 


(30) Priority: 06.05.1998 JP 12369198 




SAMSON & PARTNER 


(71) Applicant: NEC CORPORATION 




Widenmayerstrasse 5 




80538 Munchen (DE) 


Tokyo (JP) 





(54) Electronic watermark insertion device 

(57) The present invention relates to an electronic 
watermark burying device that does not bury the same 
electronic watermark data in a superimposing mode. An 
electronic watermark insertion device comprises dis- 
crete cosine transforming means (120) for subjecting an 
input image into a discrete cosine transformation; elec- 
tronic watermark burying means (130) for inserting 
electronic watermark data to data transformed by the 
discrete cosine transforming means; electronic water- 
mark detecting means (180) for detecting whether or 
not the electronic watermark data to be inserted by the 
electronic watermark burying means (130) has been 
already inserted into the input image; and selecting 



means (180) for selecting the output data from the dis- 
crete cosine transforming means (120) when the elec- 
tronic watermark detecting means (1 80) detects that the 
electronic watermark data has been already inserted 
into the input image or for selecting output data from the 
electronic watermark burying means (130) when the 
electronic watermark detecting means (180) does not 
detect that the electronic watermark data has been not 
inserted into the input image; and means (150) for sub- 
jecting data selected by the selecting means into an 
inverse discrete cosine transformation. 



FIG.1 



INPUT 
IMAGE 



[discrete 

feOSKE 



CO 

< 

<o 

LO 

m 
o> 

o 

GL 
LU 



ELECTRO K1C 
WATERMARK 
aUBYMGUMT 



ELECTRONIC 
WATERMARK 
□ATA 



ELECTRONIC 
WATERMARK 
DETECTOR 



150 

IN VERSE 
DISCRETE 
COSINE 

JKIT 



100 




OUTPUT 




IMAGE 



Printed by Xerox (UK) Business Services 
2.16.7 (HRS)/3.6 



BNSDOCID: <EP 0955767A3 I > 



EP 0 955 767 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



EP 99 10 8714 



DOCUMENTS CONSIDERED TO BE RELEVANT 




Category 


Citation of document wtth Indication, where appropriate. 
at relevant parage* 


Relevant 

tocfcHm 


eussmcATON of toe 

APPUCATtON (Int-CU) 


X 


U0 98 03014 A (PHILIPS ELECTRONICS NV 

* PuriTPC unonrw a p. kfu 
; rnlLirj NUKDcn AD V ) ) 

22 January 1998 (1998-01-22) 

* abstract * 

* column 1, line 20 - column 2, line 2 * 

* column 5, line 1 - column 5, line 17 * 


1-16 


H04N1/32 


X 


EP 0 840 513 A (NIPPON ELECTRIC CO) 
6 May 1998 (1998-05-06) 

* abstract * 

* column 8, line 9 - column 9, line 15 * 

* column 11, line 10 - column 11, line 52 
* 


1-16 




A 


EP 0 828 372 A (NIPPON ELECTRIC CO) 
11 March 1998 (1998-03-11) 

* abstract ♦ 

* column 3, line 40 - column 8, line 26 * 


1-16 




D,A 


US 5 530 759 A (HAGERLEIN KAREN A ET AL) 
25 June 1996 (1996-06-25) 

* abstract * 

JP 06 315131 A (MATSUSHITA ELECTRIC IND CO 
LTD) 8 November 1994 (1994-11-08) 

* abstract * 


1-16 




TECHNICAL R ELDS 
SEARCHED (M£U) 


D , A 


1-16 


H04N 
G06T 


D,A 


JP 05 030466 A (SONY CORP) 
5 February 1993 (1993-02-05) 
* abstract + 


1-16 




The present March report has bean drawn up for all ciakra 







THE HAGUE 



9 April 2001 



Stoffers, C 



CATEGORY OF CITED DOCUMENTS 

X : parbcuiarty relevant If taken alone 

Y : particularly relevant if comdoed wtm another 

cfcxxrnarrt of the same caftegjory 
A : toennotogtcai background 
O . non-wvtttmn dvclocure 
P : interrrcftato document 



T : tiaory or prtnapHS underlying the invention 
E : earlier patent document but publshed on, or 

after titling date 
O : document cited in the appftcaflon 
L : document cfted for other maaona 



A : member of the same patort family, exxrecpondtng 
document 



2 



EP 0 955 767 A3 



ANNEX TO THE EUROPEAN SEARCH REPORT 



Vm toUi ivr t 



PATENT APPLICATION NO. 



EP 99 10 8714 



Thte annex lists the patent fanrtfry members rotating to the patent documents rfted \n the abavs-merrtfcned E- "e-— 

The members are u contained In the European Patent Office EDF ftte on r 

The European Patent Office Is in noway BabW lor fommm partteutars wMeh are rrwrefy grven for the purpose of (rrformation. 



09-04-2001 



Patent document 
cited In scorch report 



Publication 



Patent lanHj 



PutoMcaUon 



W0 9803014 A 22-01-1998 



EP 0828372 



US 5530759 



JP 06315131 A 



JP 05030466 A 



EP 0940037 A 
JP 2000510980 T 
US 5933798 A 



EP 0840513 A 06-05-1998 



US 
AU 
AU 
CA 
JP 
SG 



5915027 A 
721462 B 
4434097 A 
2219205 A 
10145757 A 
63773 A 



11-03-1998 



US 
JP 



5848155 A 
10308867 A 



25-06-1996 



EP 
JP 



0725529 A 
8241403 A 



08-11-1994 



NONE 



05-02-1993 



NONE 



8 
o 

ui For mora (totalis about this annex see Official Journal of the European Patent Office. No. 12/82 



08-09-1999 
22-08-2000 
03-08-1999 



22-06-1999 

06- 07-2000 

07- 05-1998 
05-05-1998 

29- 05-1998 

30- 03-1999 



08-12-1998 
17-11-1998 



07-08-1996 
17-09-1996 



BNSDOCID: <EP 0955767A3_I_> 



THIS PAGE BUNK (uspto, 



